% Introduction

\chapter{Migration} % Main chapter title

\label{Chapter 5} % For referencing the chapter elsewhere, use \ref{Chapter1} 

\lhead{Chapter 5. \emph{Migration}} % This is for the header on each page - perhaps a shortened title

%\citep{Reference3}

This chapter describes the implementation of a prototype of migration of the Data Layer for the Research and Technological Collaboration Portal (RTCP) web application, following the \textbf{Data Layer migration process} described in the Chapter 3.

\section{The Proceeding}

Migrating the Data Layer of an already existing application come with various difficulties. The different issues and challenges, which have been identified in the Chapter 3, must be addressed. Thus, the procedure of migration will try to follow the roadmap provided by this thesis.

%Choosing an suitable Cloud data hosting for the Case of Study.

\subsection{Step 1: Analyzing the database of RTCP}

\textbf{Relations:} 

The class diagram in the figure \ref{fig:diagram_class} shows that the RTCP application has complex queries since it supports many to many relations. Therefore, there are queries with more than one join operation. This assumes the selection of a NoSQL database to replace the current PostgresSQL in the Cloud, would not represent an appropriated choice. Otherwise, the most common solution would be to implement a component in the Data Access Layer in order to it handle the joins.
	
\textbf{Transactions and level of consistency:} 

Since the main goal of the RTCP is provides a tools for collaboration among different institutions, such as Project management, in which different teachers and students may be working simultaneously on the same project, RTCP requires the queries are executed in isolated transactions. Thus, the cloud database should support ACID transactions. Furthermore, the user information profile must be handled carefully. These information should be accessible under strong consistency guarantees. However, there are others modules of this application which may be managed under a weaker level of consistency. In the module News, the CUIA user, who is the main user of the system, publishes news in order to share information among universities. This news could be accessible under eventual consistency since users, who read the published news, may not be seeing the last news, but it would not be a critical issue.  
	
\textbf{Database workloads:} 

Since the RTCP application was designed exclusively for private use between a number of Universities of Italy with Universities of Argentina, the support of read-heavy and write-heavy workloads by the database is not a critical issue.
	
\textbf{Size of the data handled:}

Evaluating the size of the on-premise database can be helpful to determine what hardware is required, and how much storage and instances will be needed after migration. The size of the database can be calculated in PostgresSQL using the command: 
\begin{verbatim}
SELECT pg_size_pretty(pg_database_size('database name'))
\end{verbatim}

However, at this point the following question emerge: How to know if the size of the database is small, medium or large?. According to ScaleDB \citep{Reference66}, the database size can be the interplay of following factors:

\begin{enumerate}
	\item Data Volume: The amount of data as defined by the number of records, tables, terabytes, etc. 
	
	\item Throughput: If the database is small but it services 10 million concurrent users, it will seem like a large database. Or may have a single client but it is running billions of transactions.
\end{enumerate}
 
ScaleDB \citep{Reference66} also provide the following table, which classified the database in Small, Medium and Large:

\begin{table}[!ht]
\begin{center}
    \begin{tabular}{|p{3.5cm}|p{5cm}|p{5cm}|}
		\hline 
    \textbf{Small Database}     & \textbf{Medium Database}                   & \textbf{Large Database}               \\\hline
    Fits in Memory     & Fits in Single Server             & Spread Over Multiple Servers \\\hline 
		$<$10\textsuperscript{5} Records & $<$10\textsuperscript{5} - 10\textsuperscript{7} Records  & $<$10\textsuperscript{7} Records         \\\hline 
		$<$10 GB of Data     & 10GB – 40GB of Data               & $>$40GB of Data                \\\hline 
		No Partitions      & Minimal Partitions                & Massively Sharded            \\\hline 
   \multicolumn{2}{|c|}{Performance is not a problem}                    & Performance is a Problem                         \\\hline 
    \end{tabular}
		\caption{Small, Medium and Large databases}
	\end{center}
\end{table}

The result of executing the \verb|pg_size_pretty| over the database of RTCP was less than 1 GB. Thus, the database of RTCP application may be considered as a small database that could fits in a single server.

\subsection{Step 2: Choosing a Cloud data hosting}
	
According to the previous assessment made to the current database of RTCP application, the most suitable database would be a Cloud SQL database since isolation transaction and strong consistency are required features for the most important functionalities of the system (management of Users, projects, Exchange Programs and Technology Transfer). Furthermore, the size and the current throughput of the system may be manage by SQL databases and the migration to a non-relational database could represent a high cost and effort related to the implementation of missing features, such as joins, or solve incompatibilities. However, by considering the objectives of this work, would be useful to provide a solution in which a NoSQL database is chosen to store the information of certain processes such as the News module. This will provide a wider study about the process of migration and will allow cover most of the steps described in the Chapter 3. 

\textbf{IBM Codename: BlueMix} 

Most PaaS Cloud provider enable SQL database specially adapted to run in the Cloud. Codename: BlueMix \citep{Reference56} is a PaaS offering from IBM based on open standards and cloud to build, deploy, manage and run applications like web and mobile, big data and other smart services. The main features of BlueMix are listed below:

\begin{itemize}
	\item It is built on the \textbf{Cloud Foundry} open source technology, an open source Cloud computing PaaS. 
	\item BlueMix allows developers to avoid vendor lock-in, leveraging the open and flexible cloud environment using a variety of tools from IBM, third party or open technologies.
	\item BlueMix provides a development environment to build applications incrementally composed from services.
	\item BlueMix can automatically scale a deployed application up or down based on application usage.
	\item A variety of popular programming languages and frameworks like Android, iOS, HTML/JavaScript, Java, Ruby and PHP are offered in BlueMix environment.
\end{itemize}

BlueMix provider a number of relational database management systems as a service. Some of this service are listed follow:

\begin{itemize}
	\item \textbf{SQLDB. (Owned by IBM) } This service creates a fully supported and managed database run by IBM on the IBM Cloud. SQLDB is powered by a cloud-based instance of IBM DB2 Enterprise Server Edition and provides a database service to handle the demanding workloads of large and midsize enterprise servers. DB2 databases offer industry leading performance, scale, and reliability.
	\item \textbf{MySQL database. (Owned by Cloud Foundry)}
	\item \textbf{PostgreSQL 9.2. (Owned by Cloud Foundry)}
\end{itemize}

Since the goal of this chapter is to provide a prototype of migration of the Data Layer of a case of study, BlueMix, which offers 8 GB capacity for deploying applications and also allows the use up to 20 service instances for registered IBM users, represents a good Cloud provider choice to use in the implementation of this migration. Thus, it is possible also to use the same SQL database (PostgreSQL) that currently is used in the on-premise system.

On the other hand, BlueMix allows the following NoSQL database as a service:

\begin{itemize}
	\item \textbf{MongoDB. (Owned by Cloud Foundry)} Is a open source document NoSQL database especially designed for data scaling, auto sharding, and high availability.
	\item \textbf{Cloudant JSONDB (Owned by IBM)}. It is a document-oriented NoSQL database designed to provide scaling and handles a wide variety of data types including JSON, full-text, and geospatial.
\end{itemize} 

Both NoSQL solutions provided by BlueMix allow us to take advantage of features such as high scalability and scaling. However, MongoDB could be a suitable solution to store the information of the News module since it storage system can be found in most of the Platform as a Service providers, so it will be very important in the case we need to relocate the Data layer in another provider in the future. Furthermore, MongoDB allows dynamic queries and provides a very good integrated database driver for Java. This is why MongoDB is selected for store the information of the News module.
 

\subsection{Step 3: Ensuring transparent data access to the Cloud Database layer.}

Since in step 1 it was determined that the optimum Cloud database should be a relational one to storage the most important functionalities of the system, there is no need to perform a migration of the traditional SQL data model (relational model) to another data model such as document-oriented model, key-value data model, etc. for these components of the system. Also, BlueMix allows choosing the same database that is currently used in the on-premise system (PostgreSQL). Furthermore, there is no need of dealing with incompatibilities and missing features between both on-premise and cloud versions of the database, since Cloud Foundry supports all features for PostgreSQL that are supported in the classic version. 

However, since the News module will use a NoSQL database, the cloud Database Layer will be composed by two different kind of databases, so it will be necessary to adapt the Data Access Layer in order to provide transparent access to the new database schema. 

By following the Data Access Component pattern described in the Chapter 3, the DAO package can be used as the Data Access Component to hide the complexity of accessing different storage offering. Then, the new interaction between the Business Layer and the Data Access Layer can be seen as follows:

\begin{figure}[H]
	\centering
		\includegraphics[width=0.8\textwidth]{Figures/RTCP_data_layer_and_business_layer_with_mongodb.png}
		\rule{0em}{0.5pt}
	\caption[DAO package is used as the Data Access Component to access to different storage offering.]{DAO package is used as the Data Access Component to access to different storage offering.}
	\label{fig:RTCP_data_layer_and_business_layer_with_mongodb}
\end{figure}

The DAO package will provide the same interface to the Business Layer than currently is provided by the on-premise version. The only module, inside the DAO package, that needs to be adapted is the News facade. This module uses only one table to storage the information which it manages, so we need to express this table as a MongoDB document. The News table is currently formed by 4 fields: \verb|title| (string), \verb|body| (string), \verb|date| (time-stamp), \verb|photoData| (byte string). Since MongoDB employs JSON document to define its data model, the following JSON document will be used to storage the information of the News table:

\begin{verbatim}
{
   title: "title of the news",
   body: "body of the news",
   date: "time stamp",
   photoData: "binary information of a photo",
}
\end{verbatim}

Since in the on-premise version the RTCP queries are formulated as Java objects, which are translated by JPA to the SQL language of the underlying database, in the case of the News module, the queries need to be translated by the News facade to MongoDB queries. The \verb|create| method of the on-premise version of the News facade is shown as a example in the following Java code:

\begin{lstlisting}
public class NewsFacade {

	private static final long serialVersionUID = 1L;

	@PersistenceUnit(unitName = "WebPortalPU")
	EntityManagerFactory emf;

	public NewsFacade() {
	}

	public void create(News entity) {
		EntityManager em = emf.createEntityManager();
		em.persist(entity);
		em.close();
	}
	
}
\end{lstlisting}

Now, the cloud version of the \verb|create| method is shown in the following Java code:

\begin{lstlisting}
public class NewsFacade {

	DBCollection document_news;

	public NewsFacade() {
	}

	private void connectWithMongo() {
	try {

		String host = "50.23.230.141";
		String port = "10027";
		String username = "f68835c9-e8fb-485a-b186-e140ae0dba9d";
		String password = "beba8b49-7596-401d-b72d-1bb34923de11";
		String database = "db";

		// Mongo initialization
		Mongo mongoClient = new Mongo(host, Integer.parseInt(port));
		DB db = mongoClient.getDB(database);

		if (db.authenticate(username, password.toCharArray())) {
			document_news = db.getCollection("Newscollection");
		} else {
			throw new Exception("Authentication Failed");
		}
	} catch (Exception e) {
		e.printStackTrace();
	}
	}

	public void create(News news) {
		connectWithMongo();
		BasicDBObject document = new BasicDBObject();
		document.put("id", news.getId());
		document.put("title", news.getTitle());
		document.put("body", news.getBody());
		document.put("date", news.getDateTime());
		document.put("photoData", news.getPhotoData());
		document_news.insert(document);
	}

}

\end{lstlisting}

The above implementation of the \verb|create| method shows the same interface as before, so it hides the complexity of managing two different databases. 

%Since in step 1 it was determined that the optimum Cloud database should be a relational one, there is no need to perform a migration of the traditional SQL data model (relational model) to another data model such as document-oriented model, key-value data model, etc. Also, BlueMix allows choosing the same database that is currently used in the on-premise system (PostgreSQL). Furthermore, there is no need of dealing with incompatibilities and missing features between both on-premise and cloud versions of the database, since Cloud Foundry supports all features for PostgreSQL that are supported in the classic version. 

%Beyond the selected cloud database was a relational one, would be helpful to analyze a scenario in which a NoSQL database is choose to store only the information of the News module. In this case, since the the Database Layer will be composed by two kind of different databases, it will be necessary to adapt the Data Access Layer in order to provide transparent access to the new database schema.  

\subsection{Step 4: Converting the Data Layer in a loose coupling component.}

In the Chapter 3, the importance of a weak-coupling between the Data Layer and the Business Layer was described. To implement loose coupling between components, either a Message-oriented-middleware implementation or the Enterprise Service Bus architecture can be used, since these can implement the communication between interacting software application components. Another solution to provide loose coupling is using Web Services which follow the Services Oriented Architecture (SOA). Thus, Web services enabling location transparency by utilizing UDDI (Universal Description, Discovery and Integration) register for runtime discovery \citep{Reference69}. Clients can locate the desired service dynamically by requesting the service from the registry. The Web services architecture provides benefits of loose coupling by providing a mechanism to locate, bind, and call the service dynamically. Web services allow service developers to implement services using any programming language such as Java or .NET. Web services clients communicate to the service using SOAP, which is XML over HTTP. In addition, Web services developers need not worry about wire-protocol issues since the SOA platforms handle these issues for them.

%\textbf{Enterprise Service Bus}
%
%Enterprise Service Bus is used in Distributed Computing to implements the communication between software components in a Software-Oriented Architecture (SOA). Thus, its primary use is in application integration \citep{Reference57}. The creation of the ESB was driven mainly by motivation to find a standard, structured and general purpose concept for describing implementation of loosely-coupled software components (called: services) that are expected to be independently deployed, running, heterogeneous and disparate within a network \citep{Reference57}. 
%
%\begin{figure} 
	%\centering
		%\includegraphics[width=0.8\textwidth]{Figures/esb.png}
		%\rule{0em}{0.5pt}
	%\caption[ESB as the common communications medium.]{ESB as the common communications medium.}
	%\label{fig:esb}
%\end{figure}
%
%The ESB enables software applications running on different platforms, written in different programming languages, and using different programming models, to communicate with each other. Messages can sent through the bus ``speaking'' the language of one application and ``get off'' the bus speaking the language of another.

\textbf{The solution to enable loose coupling}

%By taking advantages of the ESB, in order to provide loose coupling between the Data Layer and Business Layer, the following solution will be implemented:
In order to provide loose coupling between the Data Layer and Business Layer, the following solution will be implemented:

\begin{enumerate}
	\item Each component (facades) belonging to the DAO module will be converted into individual web services.
	\item The Business Layer requests will be done through SOAP request messages.
	%\item An ESB will provide the remote communication between both layers.
\end{enumerate}

\begin{figure}[H]
	\centering
		\includegraphics[width=0.8\textwidth]{Figures/loose_coupled_data_layer2.png}
		\rule{0em}{0.5pt}
	\caption[RTCP Data layer implemented as a loose coupled component.]{RTCP Data layer implemented as a loose coupled component.}
	\label{fig:loose_coupled_data_layer}
\end{figure}

\textbf{EJB Web Services}

In order to convert the DAO components into individual web services, it is possible to use the Enterprise Java Beans (EJB) technology, since it provides options to expose a session EJB as a service. \verb|@WebService| annotation is used to mark a class as a web service end point and \verb|@WebMethod| is used to expose a method as web method to client. The following Java code provides a example in which the DAO component \verb|UserFacade| is enabled as web service.

\begin{lstlisting}
@Stateless
@WebService(serviceName="UsersFacadeWS")
public class UsersFacade {

	@PersistenceUnit(unitName = "WebPortalPU")
	EntityManagerFactory emf;

	public UsersFacade() {
	}

	@WebMethod(operationName = "create")
	public void create(Users entity) {
		EntityManager em = emf.createEntityManager();
		em.persist(entity);
		em.close();
	}

	@WebMethod(operationName = "edit")
	public void edit(Users entity) {
		EntityManager em = emf.createEntityManager();
		em.merge(entity);
		em.close();
	}

	@WebMethod(operationName = "remove")
	public void remove(Users entity) {
		EntityManager em = emf.createEntityManager();
		em.remove(em.merge(entity));
		em.close();
	}

	@WebMethod(operationName = "find")
	public Users find(Object id) {
		EntityManager em = emf.createEntityManager();
		return em.find(Users.class, id);
	}
	
	//Other Web Methods

}
\end{lstlisting}

The above Java code shows some operations such as \verb|create, edit, remove and find|, which are exposed as a services. These services will be consumed by the Business Layer to perform the their business operations.

\subsection{Step 5: Ensuring elasticity and scalability.}

Other important step in process of adapting application components, that must be migrated to the Cloud, is to make them stateless components. This means the component do not maintain an internal state that could hinder the scalability. Instead, if a component needs to maintain an internal state among different instances of it, different techniques can be applied. Since the components do not maintain an internal state, new instances of it can be created in an elastic way to attend concurrent requests.  

The Enterprise Java Beans (EJB) technology also allows to define stateless components. In the previous section, the DAO component \verb|UserFacade| is defined as a \textbf{stateless Java session bean}. A stateless bean have no state held on behalf of the client and only need to exist for the duration of a single method call. The container can pool multiple bean instances and grab the most convenient one when a client needs it. 

In order to clarified the functionality of a stateless session bean y how does it can provide scalability and rapid elasticity, it is necessary to take a look of the Life Cycle of a stateless session bean.

\begin{figure}[H]
	\centering
		\includegraphics[width=0.5\textwidth]{Figures/stateless_beans_file_cycle.png}
		\rule{0em}{0.5pt}
	\caption[Life Cycle of a Stateless Session Bean.]{Life Cycle of a Stateless Session Bean.}
	\label{fig:stateless_beans_file_cycle}
\end{figure}

The figure \ref{fig:stateless_beans_file_cycle} shows the Life Cycle of a Stateless Session Bean. Firstly, when a bean instance is in the Does Not Exist state, it is not an instance in the memory of the system. Stateless bean instances enter the Method-Ready Pool as the container needs them. When the EJB server is first started, it will probably create a number of stateless bean instances and enter them into the Method-Ready Pool. When the number of stateless instances servicing client requests is insufficient, more can be created and added to the pool. The instances can maintain open connections to resources for their entire life cycle until them are closed \citep{Reference68}. 

\subsection{Step 6: Ensuring the level of data consistency required.}

The goal of this step is to ensure the level of consistency required by the system is achieved. In the step 1 it was determined that most functionalities of the system need to be accessible under strong consistency with the exception of the News module. In this case, the consistency level is ensured itself by the storage systems that have been selected. The relational database will enable isolated transactions, strong consistency for reading and durability for writings, which are features required by the RTCP application. On the other hand, MongoDB will allow to take advantage of the benefits of eventual consistency reducing the waiting times for displaying the news and allowing high availability.   



